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Abstract 

Although recent neurophysiological experiments suggest that synchronous neural activity is in- 
volved in some perceptual and cognitive processes, the functional role of such coherent neuronal 
behavior is not well understood. As a first step in clarifying this role, we investigate how the tem- 
poral coherence of certain neuronal activity affects the activity pattern in a neural network. Using 
a simple network of leaky integrate- and- fire neurons, we study the effects of synchronized incoming 
spikes on the functioning of two mechanisms typically used in model neural systems, winner-take- 
all competition and associative memory. We demonstrate that a pair of switches undergone by 
the incoming spikes, from asynchronous to synchronous and then back to asynchronous, triggers a 
transition of the network from one state to another state. In the case of associative memory, for 
example, this switching controls the timing of the next recalling, whereas the firing rate pattern in 
the asynchronous state prepares the network for the next retrieval pattern. 

PACS numbers: 87.19.La, 05.45.Xt, 87.18.Hf, 87.18. Sn 
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I. INTRODUCTION 



Synchronous neural activity, that is, coherent spiking behavior, is widely observed in var- 
ious cortical areas, and is often stimulus-specific and task-specific jlllsl- These observations 
suggest that synchronous neural activity may play a significant functional role in cognitive 
and perceptual processes 0, |^ . One proposed possibility is that such a coherent spiking is 
the signal for the binding of different attributes of an object into a single percept. To this 
hypothesis, we have to first clarify the neural mechanism underlying this coherent activity. 
Over the last decade, a considerable number of studies have been focused on determining the 
dynamical mechanisms underlying synchronous firing in neural systems Q, Q, S Q Q 
, for example, elucidating the neural mechanisms that enable cortical neurons to synchro- 
nize and how the neural network controls the degree of synchrony in a stimulus-dependent 



manner. 
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131 ] The next important issue is to clarify how such coherent activity contributes 



to information processing. However, there have been relatively few studies addressing this 
issue. Q,Q Considering the above situation, in this paper, we explore the possible func- 
tional role of synchronized neural activity. 

Many electrophysiological experiments have shown that cortical neurons do not always 
synchronize and that the degree of synchrony is dynamically modulated by a number of 
factors, such as external stimuli and the internal state. In fact, recent experiments show that 
the degree of coherence among; 

is observed that such coherent activity transiently lasts for a few hundred msec. These facts 
indicate that the switching between asynchrony and synchrony provides a crucial signal for 
information processes in the brain. In developing a theory of information processing in the 
brain, it is necessary to understand how modulation of the level of synchrony influences the 
function of the neural network. In particular, we study the influence of synchrony- asynchrony 
switching in incoming spikes on the activity pattern on a neural network, resulting from two 
mechanisms typically employed in model neural networks, winner-take-all competition and 
associative memory. 



2 



II. WINNER- TAKE- ALL COMPETITION MODEL 



Let us begin our analysis by considering winner-take-all(WTA) competition, which is com- 
monly used as a computational model to determine the best choice among different possible 
outputs of the network. Is is believed that this competition mechanism provides a funda- 
mental mechanism used in the brain. For example, various models using this mechanism 
have successfully reproduced columnar structures found in the visual cortex 17, 18, 19, 2o| . 
Through WTA competition, the only neuron receiving the largest input in a network be- 
comes active as the 'winner'. Recent simulations have revealed that such competition is 
extremely sensitive to the coherence among neuronal action potentials [21]. We study this 
phenomenon theoretically and explore the possible role of synchronization in the functioning 
of neural networks. 

A schematic of our WTA model is displayed in Figure ^a). As the single neuron model, 
we adopt the leaky integrate-and-fire neuron model, in which the basic equation determining 
the membrane potential Vi of the z-th neuron is given by 

rj^- = -(Vi - V vest ) + R m h(t), (2) 

where R m = 100MQ and r m = R m C m = 20ms are the membrane input resistance and 
the membrane time constant, respectively. Whenever Vi(t) reaches the threshold voltage 
Vth = — 50mV, a spike is generated, and Vt(t) is instantaneously reset to the resting potential, 
V Tes t = — 65mV. The total current Uit) consists of the sum of all inputs to the neuron, given 
by 



Iiit) = if h (t) + if lfex (t) + py*-™y n ( t ) + I° xt (t), (3) 

/inh 

w 



with i^(i) = -^££j(*-i,-A) 



j^i spike 

/• elfeX =felfex^J(t-t i -A), 
spike 

where N is the total number of neurons, tj is the time at which the i-th neuron generates 
a spike, and A is the synaptic transmission delay. The postsynaptic potential is expressed 
simply as the alpha function J(t), which is given by 

J(t) = Aexp(— )6(t), 
r z t 
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FIG. 1: (a) Schematic illustration of the winner-take-all(WTA) competition model. Each neuron 
receives two external inputs, one from the asynchronous/synchronous input layer always uniformly 
inputs and one from the pattern input layer neuron-specific inputs, (b) WTA competition sup- 
pressed by synchronized incoming spikes. The upper two graphs display a firing pattern and a 
rastergram of the output layer. The firing pattern of the output layer is shown as a lOxlO-dot 
image, in which the gray level of a dot indicates the averaged firing rate over 200 ms (black under- 
bar), normalized by dividing by the maximum firing rate. In the lower two graphs, the presentation 
of the pattern input layer and rastergram of the synchronous/asynchronous input layer are shown. 
Pattern input is presented as a lOxlO-dot image in which a black dot indicates active state and a 
white dot indicates silent state, see detail right images. For an asynchronous input, the network 
exhibits typical behavior in the case of WTA competition. The winner pattern A is maintained 
even if a new pattern B is presented in the pattern input layer. However, a brief synchronous input 
causes the WTA competition to reset, leading to a transition to a new winner pattern B. 

where r is the time constant of a synapse, and 0(£) is the step function, given by 




The current /- nh (t) represents the effect of all-to-all uniform inhibitory couplings, which 
is responsible for the WTA behavior of the network. We also assume that each neuron 
is self-connected by an excitatory synapse, lf elfex (t). This single neuron in our model can 
be regarded as representing a population of the real neurons comprising a single columnar 
structure. Therefore, it is quite natural that such excitatory self-connections are introduced, 
corresponding to the mutual excitatory synaptic connections existing within an actual col- 
umn of this kind. 

We assume that each neuron in the network receives input from two types of external 
sources. One is the synchronous/asynchronous (Syn-Asyn) input layer, in which the level of 
coherence among the input spikes can be controlled to examine the effect of the synchrony- 
asynchrony switching of incoming spikes on the activity pattern in a neural network. This 
uniformly applied current is generated according to 



input layer and A^ nput is the total number of neurons in this input layer. We consider 
the situation in which this input layer possesses two modes, an asynchronous mode and 
a synchronous mode. In the asynchronous mode, each spike is generated by a Poisson 
process. In the synchronous mode, 45 percent of the neurons in this layer fire periodically 
and synchronously with spike timings distributed in a Gaussian distribution of variance 
a = 4ms. In both modes, we assume that the average firing rate is 25Hz. 

The other external source is the pattern input layer, which is the neuron-specific current, 
given by 



Each neuron in this layer projects to one neuron in the output layer in a one-to-one corre- 
spondence. By controlling the activity of neurons in this layer, the biased current for each 
neuron in the output layer is determined, which affects the firing pattern selected through 
the WTA competition. 



For these synaptic connections and external inputs, we use the parameter values A = 1ms, 
t = 4ms, g inh = 4.8nS,g scl{cx = 3.0nS, g input = 7.UnS, N input = 1000 and AI = 7. 5p -A. 




where t syn asyn is the time at which neurons receive a spike from a neuron in the Syn-Asyn 




AI active state , 







silent state. 
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Figure W[b) depicts a typical effect of synchronized incoming spikes on the properties of 
the WTA competition. Here the simulation was performed with a network of 100 neurons. 
The system was started with the condition that the Syn-Asyn input layer is set in asyn- 
chronous mode, and in the pattern input layer, the currents / 4 ext (t) in the 1st through 40th 
neurons are active ( Pattern 'A'). Owing to the nature of the WTA competition, only those 
neurons that receive a larger input are active (as seen during the first 200 ms in FigOJb)). 
After these currents in the pattern input layer turn off, the firing of the winners(Pattern 
'A') is maintained. Next, even though another current pattern 'B' applying to some loser 
neurons(60-100th) is presented in the input layer, the firing pattern of winners remains stable 
(time steps 400ms - 600ms). This result is typical behavior of WTA competition models. 

With the situation as described above, let us consider a transient synchronized spike 
inputs with temporal switching modes in the Syn-Asyn input layer. In this shown 
in Figure IHb), the synchronized incoming spikes cause the old firing pattern to become 
unstable. Although the old firing pattern 'A' is maintained at first, the network eventually 
exhibits a collective synchronized oscillatory state. When the asynchronous mode in the 
Syn-Asyn input layer is recovered, a second firing pattern 'B' is realized through WTA 
competition, as determined by the activity of the pattern input layer. The other neurons(40- 
60th) that receive no pattern input do not fire even in a synchronous input period. It 
depends on the parameters whether or not these neurons fire without pattern input during 
synchronous input period. However, even if these neuron fire, the next pattern 'B' always 
is realized in the following asynchronous input period. Therefore, it should be noted that 
a transition of the firing pattern in the output layer is induced by synchrony- asynchrony 
switching in incoming spikes. This result means that the synchronized incoming spikes 
nullify the effect of the function of WTA competition, erasing the current pattern realized 
in the WTA competition, and thereby allowing it to make a transition to a new pattern 
prepared by the pattern input layer. 

III. ANALYSIS 

In order to understand the result of the numerical simulations described above, we now 
analyze the stability of the state selected through the WTA competition. 

For the convenience of analysis, first we rescale equation (J2J). Using a transformation 
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given by 



s = t/r m 



equation |21 is rewritten as 



= -«, + !,(*), (4) 



where parameters are rescaled as 

a a / / a t A/.R m g x R n 
A = A/r m , r = r/r m , AI = — — — , and g x 



V\h ~ K-est (Vth — ^rest)Tm 

Next, assuming that the total number of neurons in these layers is sufficiently large and 
the variance of the synchronous spike timing a = 0, the form of the input currents can be 
simplified as 



.syn-asyn^ 



^input/To asynchronous, 
9m P nt{p J2n J ( s ~ nT o) + ( l ~ p)/ t o) synchronous, 



where T is the period of the input spikes from Syn-Asyn input layer, p is the ratio of 
synchronized neurons. 

From the results of simulations, we know that typically, winner neurons fire periodically 
and synchronously at the times Sj = nT(n = 1, 2, ...) for the asynchronous mode in the Syn- 
Asyn input layer and Sj = n(T — 9) for the synchronous mode. Consequently, we consider 
only these solutions in this analysis and do not deal with other more complicated solutions. 
To analyze these solutions, the unknown parameters T and 9 should be determined in a self- 
consistent way. In the asynchronous mode, integrating equation 0] between nT and (n + l)T, 
we get a self-consistent equation for the period T, 



1 = {h + " e- T ) + (to - 9L)K T (-A) (5) 

-to 

K T (s) = e~ T [ e u Y,J(s + n + mT)du, g' mh = 9i j3^1 (6 ) 

where N p is the number of the winner neurons. Solving this equation, T is determined and 
the spike trains of winner neurons are determined. 
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n+1 



Similarly, in synchronous mode, we also derive a self-consistent equation for 6, 

1 = (j. + - e- To ) + ( felfex - ^ nh )K To (-A) + pg mput K To (9T ), (7) 

Jo 

and determine the spike trains of winner neurons. 

Given the spike trains of the winner neurons, the time evolution of the loser neurons 
can be calculated. Let f "(</») denote the membrane potential of z-th neuron at the time 
s = T{n + 0), in which <p goes from to 1. The return map from to is given by 

e~ T v% + {k + ^)(1 - e- T ) - g' iQh K T {(j)T - A) asynchronous, 
e" T ^ + (I { + (1 - p jf input )(l - e~ T °) (8) 
-9Lh K T ((</> - 0)T - A) + pg input K To (<pT Q ) synchronous. 

v^(<j>) converges to the limit t>°°(0), 

{(U+^)(l-e- T )-g' inh K T (<pT-A) 
2 i- e -T asynchronous 
(1 - p ^"P"t {1 _ e -T 0) _ g , nhK (^-e)T -A)+pg input K To (4>T ) ^ 
Q i- e ~ T o synchronous 

The stability of the WTA state is ensured by the condition that none of the loser neurons 
fire, therefore, 

v°°(4>) < 1 in [0, 1) for all loser neurons i . (10) 

As depicted in Figure Wlb) corresponding to t — 400 — 600 ms, we consider the situation 
in which the selected neurons remain active without a biased input and some loser neurons 
receive a biased current (AJ) from the pattern input layer. Using Eq llOl in the situation 
described above, we determine the condition under which the firing state of the selected 
neurons remains stable and the neurons receiving the biased input still cannot fire. The 
stable parameter regions obtained in this manner for the cases of the synchronous and 
asynchronous modes are displayed in Figure El 

In the shaded regions, destabilization of the WTA state occurs in response to the switching 
of the mode of the Syn-Asyn input layer from asynchronous to synchronous incoming spikes. 
This destabilization leads to a transition of the neuronal state into the next firing pattern, 
as determined by the pattern input layer. Thus, if the parameter within the shaded region is 
used, the WTA network exhibits synchrony-induced switching behavior. The validity of this 
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FIG. 2: Two-parameter phase diagrams concerning the stability of the state selected through 
the winner-take-all competition. This state is stable if the parameter value is located below the 
solid line (dashed line) for the case of asynchronous (synchronous) incoming spikes. In the shaded 
regions, destabilization of the WTA state occurs in response to the switching from asynchronous to 
synchronous incoming spikes. The data points indicate simulation results, (a) Dependence of the 
stability on the synaptic transmission delay, A, and the biased current from pattern input layer, 
AI. (b) Same as (a), except that the horizontal axis corresponds to the synaptic time constant, r. 
The parameter values used in Figure Q is indicated by the +. 

analysis is shown by the numerical simulation data in Fig|21 In addition, it also shows that 
the result is not too sensitive to both the synaptic delayA and the synaptic time constant 



IV. ASSOCIATIVE MEMORY 

We next consider the case of the associative memory model 1313' * n which information 
is stored in the synaptic weight matrix, Wq. For the learning rule of the synaptic matrix, 
we adopt the Willshaw-type learning rule [24J, defined by 



where P is the total number of the stored patterns, Q represents the state in the i-th neuron 
in the /i-th pattern and takes only two values 1 (firing) and (quiescent). Replacing the 
self-excitatory coupling in the WTA model with the synaptic matrix Eq. the resultant 



T. 




(11) 
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dynamics for the i-th neuron are described by 

rj^ = -(Vi - V rest ) + + ir(t) + I syn ~ asyn + ir\t), (12) 

jy^i spike 

where the factor N = ■ Wij)i is the average connections of synaptic matrix Wij. 

Figure 01 depicts typical behavior of the associative memory model found in our numerical 
simulations, in which 10 patterns ('x' , V and 8 randomly generated patterns) are stored 
in the network of 1600 neurons. The 'key' pattern is presented as the activity in the pattern 
input layer, displayed as a 40x40-dot image. The 'retrieval' pattern is displayed as a 40x40- 
dot image, in which the gray level of a dot indicates the averaged firing rate over 100 ms, 
normalized by dividing by the maximum firing rate. As shown in Figure when a slightly 
distorted version of the pattern 'x' , consisting of 'x' plus some noise, is presented as a 
key pattern, the network correctly realizes the pattern ' x ' as the outcome of the associative 
memory functioning (0 - 200ms). It is interesting that, if the spike trains in the Syn-Asyn 
input layer synchronize, the existing retrieval pattern 'x' becomes unstable(600 - 900ms), 
leading eventually to the synchronous oscillatory state as in the case of the WTA model. 
Then, when the input spike trains become uncorrelated again, the network comes to exhibit 
the new retrieval pattern V in response to the key pattern presented by the pattern input 
layer (900 - 1200ms). Hence, the synchrony-asynchrony switching of the incoming spikes 
plays a crucial role in triggering a change of the retrieval pattern, whereas the biased input 
from the pattern input layer prepares the network for the next retrieval pattern. 



V. DISCUSSIONS 



In the present paper, we have studied the manner in which the spike synchrony in the 
input layer can affect the activity pattern in a neural network. We find that the synchrony- 
asynchrony switching undergone by the incoming spikes plays a key role in triggering a 
transition from one state to another state, even if the firing rate of incoming spikes is kept 
constant. For example, in the case of the associative memory model, the timing of the 
next retrieval can be controlled by such a synchrony-asynchrony switching of the external 
uniform input spikes, whereas the averaged rate pattern in the pattern input layer allows 
for the selection of the set of firing neurons for the next retrieval pattern. 
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FIG. 3: A possible role of synchronized incoming spikes in controlling the timing of the next 
retrieval in associative memory. The transition of the retrieval pattern is induced by the synchrony- 
asynchrony switching of the incoming spikes in the associative memory model. The upper two 
graphs display the pattern of the normalized firing rate presented as 40x40-dot images and a 
rastergram of the output layer. The lower two graphs display the presentation of the pattern input 
layer (key pattern) and the rastergram of sync/async input layer. 

We will comment on a conventional way to reset activity patterns in WTA network. 
Because of the well-known multi-stability in a WTA network, the transition in the output 
layer can also be achieved by changing the biased current from the pattern input layer. 
Therefore in Figure ^b), if the input current from the pattern input layer for pattern B 
is sufficiently large, a switching from A to B occurs. This increase in the biased current 
corresponds to an increasing AJ in Figure |2l which makes the WTA state unstable. By 
contrast with this conventional way, the important point of our model is that the firing rate 
of incoming spikes does not change during a transition in the output layer. The purpose of 
the present paper is to propose an alternative mechanism to control the neuronal activity 
in a neural network by altering incoming spike synchrony only. 

Some multi-recording experiments suggest that spike synchronization preferentially oc- 
curs in relation to a purely cognitive event, where firing rate modulations were absent ll. |25| . 
For example, it is shown that before the onset of voluntary movement, cortical neurons in 
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the motor area transiently synchronize without changing the firing rate of each neuron. In 
addition, many studies in the recording of local field potential or electro-encephalographic 
activities also show that 7 frequency oscillation is observed in association with upcoming 
movements. Therefore, some have suggested that these synchronous neural activities can 
be regarded as the representation of an anticipation or a preparation of an upcoming move- 
ment. In our understanding, such a synchronization in neural activity seems to play a role 
in triggering a transition to the next firing state for the upcoming movement. Hence, we 
believe that some of the observed synchronous activities are essentially relevant to our result 
in the present paper. 

Next, we will comment on the fact that the same transition can be achieved by uniform 
rate modulation in the Syn-Asyn input layer. From a dynamical point of view, synchrony- 
asynchrony switching constitutes a kind of perturbation leading to a transition from one 
attractor to another. It can be imagined that the same effect could be achieved through 
some other kinds of perturbations that do not involve synchrony among spikes, such as 
uniform modulation of the firing rate. To clarify this point, we investigated the effectiveness 
of such rate-modulated spikes numerically. In this case, instead of the synchronized incoming 
spikes, we used an abrupt change in the firing rate from 25Hz to 75Hz or 100Hz, but the 
timing of the change in firing rate is distributed according to a Gaussian distribution with 
variance a (Figure HJi). Figure EJo displays the probability of a successful transition induced 
by such a modulation of the firing rate as a function of the width of the variance a. We 
see that a change to 75Hz is insufficient to achieve a high probability of transition, even 
when the changes in firing rates are completely synchronous (i.e. cr=0). In the case that the 
firing rate is increased to 100Hz and a is sufficiently small, the probability for a transition 
is comparable to the case of synchronized spikes (see Sync.25Hz in Fig0)3). Therefore, we 
conclude that a large and simultaneous increase in firing rate is required to realize a high 
probability for a transition. This implies that this modulation can practically be regarded 
kind of temporal coding. 

Finally, we will point out that the synchrony-induced transition has different properties 
from the rate-modulation-induced one as mentioned above. As shown in Fig^c), the WTA 
state is still stable even with a high firing rate. This analysis is based on the assumption 
that the incoming spike rate is constant. Therefore, a sudden increase in the firing rate may 
cause a transition to another firing pattern. In fact, the numerical simulations show that a 
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Input Spike &»e(Hz) 

FIG. 4: (a) Instead of synchronized spikes, a rate-modulated perturbation is applied to the neural 
network by changing the firing rate from 25Hz to 75 or 100Hz. The timing of the rate switching 
is distributed in a Gaussian distribution with variance a. (b) The probability of a transition of 
the firing pattern as a function of the fluctuation associated with the rate switching (solid curves). 
The dashed curves corresponds to the case of synchronized spikes, (c) Dependence of the stability 
of the state selected through the winner-take-all competition on the input spike rate in Syn-Asyn 
input layer and the pattern input current AI. Note that the constant rate for input is assumed. 

large and almost simultaneous increase in firing rate over all neurons can trigger a transition 
in the output layer. On the other hand, by switching to a synchronous spike input, the WTA 
state becomes unstable. Therefore, the old firing pattern of the WTA state is eventually 
broken for the several synchronous spikes, whereas the old pattern is still maintained for the 
first synchronous spike (Fig^b)). This result indicate that synchrony- induced transition 
mechanism is different from the rate-modulation-induced method. We believe that instead 
of rate-modulated type of perturbations, synchrony-asynchrony switching in incoming spikes 
may provide another sophisticated way to cause a transition of the firing pattern. 
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